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Description 



[0001] The present invention relates to an object monitoring system and, in particular, to a system for monitoring 
vehicles. 

[0002] Authorities responsible for traffic management and the laws which govern the use of vehicles require systems 
which can monitor traffic continuously and detect breaches of the law, without requiring the expense of having personnel 
present at the scene of the infringement. Systems which are able to monitor a large number of locations, detect in- 
fringements and issue infringement notices are particularly advantageous as they relieve personnel, such as police, 
from the task of traffic management and allow them to pursue other tasks. By continuously monitoring a location the 
systems also act as a deterrent to infringers and may assist in reducing accidents which cause road fatalities and 
casualties. It would also be advantageous to be able to monitor road usage in order to make decisions on road damage 
caused by heavy vehicles. 

[0003] A number of traffic management systems are presently in use, such as speed cameras and red light cameras 
for road traffic. The known systems employ cameras which are triggered when an infringement is detected. Optical 
sensors placed on the side of the road, pressure sensors placed underneath the road and radar signals reflected from 
vehicles are used to detect the presence of a vehicle and determine infringement. The sensors and radar signals are 
used to generate a trigger signal to activate a camera to take a picture of the vehicle which includes details from which 
the vehicle can be identified, such as a car licence plate. Use of road based sensors is disadvantageous as they require 
the road to be altered or excavated for installation or, when placed on the side of the road, can be easily detected and 
damaged. Also electrical cabling needs to be installed and connected between the sensors and the camera. The use 
of electromagnetic signals which are transmitted to and reflected from a vehicle, such as radar signals, to detect pres- 
ence and infringement is also disadvantageous as these signals can be detected by detection units placed in a vehicle 
to alert the driver as to their presence. 

[0004] It is advantageous therefore to provide a system which can detect vehicle presence and infringement without 
transmitting any electromagnetic signals or using road based sensors. 

[0005] The cameras presently in use also use photographic film which has the disadvantage that it needs to be 
continually replaced at the location of the camera. Accordingly, a number of red light cameras in metropolitan areas 
do not always include film and do not continuously monitor the corresponding intersection. 

[0006] Speed detection systems which use only cameras are described in a number of publications. The systems 
are able to monitor traffic flow and detect instantaneous speed infringements but the systems are relatively limited with 
respect to the information they can obtain on a vehicle whilst it is being monitored, and the systems are also unable 
to selectively acquire information on specified vehicle types. 

[0007] The pictures or images acquired by the camera also normally need to be examined by personnel to extract 
the information to identify the vehicle and determine the person responsible for it, which is a time consuming process. 
If the image could be processed within a relatively short time of acquisition then it could be used as a basis for alerting 
authorities in the region to seek and hold the vehicle, for example, if the information identifies it as being stolen. Ac- 
cordingly, it would be advantageous to provide a system which can process images in real time to obtain detailed 
information on a vehicle and issue alert information and infringement notices without requiring human intervention. 
[0008] When travelling a long distance, vehicle users, in particular truck drivers, tend to transgress speed limits so 
as to shorten the time in travelling to the destination and during the journey their speed may vary from a range which 
is under the limit to one which exceeds the limit. The known systems for detecting speed infringement concentrate on 
detecting the instantaneous speed of a vehicle at a particular location and therefore depending on the location at which 
the detection unit is placed, it may not detect users who infringe sporadically over a long distance. Also truck and bus 
drivers who exceed a recommended time of travel by avoiding rest stops and inaccurately complete log books may 
not be detected. Hence, it would be advantageous to provide a system which can detect the average speed of a vehicle 
over a relatively long distance. It is also advantageous to provide a system which can monitor vehicles in more than 
one lane of a multi-lane carriageway. 

[0009] GB-A-2 227 589 discloses a camera system for acquisition and adjustment of a static object triggered by 
means of a photocell detector unit placed at a given location detecting the presence of the object. The system does 
not monitor movement of an object, but merely detects its presence. 

[0010]. EP-A-0 339 988 discloses a traffic monitoring apparatus having a transceiver assembly that emits beams of 
radiation that are reflected back to the transceiver assembly by reflectors located in a thoroughfare. Interruptions of 
the beams by a vehicle and the space in between them are used to determine the presence of a vehicle and sense 
the speed of it. If a speed violation occurs, a video camera is triggered to acquire an image of the field of view of the 
camera which hopefully will include the vehicle. 
[001 1] The invention is achieved as set out in the appended claims. 

[001 2] According to the present invention there is provided an object monitoring system comprising camera means, 
characterised in that the camera means is adapted to monitor movement of an object to predetermine, based on the 
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monitored movement of the object, an acquisition time at which an'image can be acquired at a predetermined position 
of said object relative to said camera means, and to acquire an image at the predetermined acquisition time and the 
predetermined position. 

[0013] The present invention also provides an object monitoring system comprising camera means for monitoring 
5 moving objects, and image processing means, responsive to said camera means, for detecting a predetermined moving 
object from other moving and static objects. 

[0014] The present invention further provides an object monitoring system comprising camera means for tracking 
and acquiring an image of a moving object from which information identifying said object can be automatically extracted. 
[001 S] Preferably said system includes means for transmitting said image over a digital telecommunications network. 

w [001 6] The present invention also provides a vehicle monitoring system, comprising camera means for continuously 
detecting and tracking moving vehicles over a multi-lane carriageway, and acquiring images of predetermined vehicles 
at an acquisition area on said carriageway from which identifying information on said vehicles can be extracted. 
[0017] The present invention also provides a vehicle monitoring system comprising camera means for monitoring 
moving vehicles to determine if said vehicle is of a predetermined type and, in response thereto, capturing respective 

15 images of vehicles of said predetermined type. 

[001 8] The present invention further provides a vehicle monitoring system comprising camera means for monitoring 
a vehicle to detect a law infringement and determine a predetermined time to acquire an image of said vehicle, and 
for capturing an image of said vehicle at said predetermined time in response to detecting said infringement. 
[0019] The present invention also provides a vehicle monitoring system comprising camera means for monitoring 

20 vehicles on a roadway, discriminating between large vehicles, such as trucks and buses, and small vehicles, such as 
cars, on said roadway so as to acquire images of only the large vehicles from which vehicle information can be obtained. 
[0020] A preferred embodiment of the present invention is hereinafter described, by way of example only, with ref- 
erence to the accompanying drawings wherein: 

25 Figures 1 to 3 are side views illustrating use of a preferred system for monitoring vehicles; 

Figure 4 is a front perspective view illustrating use of a preferred system for monitoring vehicles; 

Figure 5 is a block diagram of a preferred embodiment of the vehicle monitoring system; 

Figure 6 is a block diagram of connection across a digital telecommunications network of two nodes and a central 

server of the vehicle monitoring system; 
30 Figure 7 is a view illustrating connection of a large number of nodes of the vehicle monitoring system; 

Figure 8 is a block diagram of vehicle detection and image capture circuitry of the vehicle monitoring system; 

Figure 9 is a digitised image produced by the vehicle detection circuitry from an image generated by a detection 

camera of the system; 

Figure 10 is a block diagram of the control of the circuit boards of the vehicle detection circuitry to perform a 
35 segmentation process; 

Figure 11 is a static background image stored in the vehicle detection circuitry; 
Figure 12 is a difference image generated by the vehicle detection circuitry; 

Figure 13 Is an image illustrating regions of shadow which are filtered from the image obtained by the detection 
camera; 

40 Figure 14 is a segmented image derived by the vehicle detection circuitry; 

Figure 1 5 is a histogram of pixel grey levels; 

Figure 1 6 is a real time status display generated by the system; 

Figure 17 is a flow diagram illustrating flow between the software tasks of the system; 

Figure 18 is a diagram of the formation of "black triangles" in a processing window of the system; 
45 Figure 1 9 is a diagram illustrating measurement of coverage of blob regions produced by the system; 

Figure 20 is a diagram illustrating vertical extension of blob regions to perform clusters; 

Figure 21 is a graph of extension amounts which are stored in a look-up table of the system; 

Figure 22 is a diagram illustrating extension based on blob region width; 

Figure 23 is a diagram of overlap detection for clusters produced by the system; 
so Figure 24 is a diagram illustrating a labelling method performed by the system; 

Figure 25 is a diagram of the roadway coordinates used by the system; 

Figure 26 is a graph of the trajectory of clusters; 

Figure 27 is a graph of the trajectory of clusters transformed to the roadway coordinates; 
Figure 28 is a diagram of data values obtained by trajectory software of the system; 
55 Figure 29 is a block diagram of a timing control board of the system; 

Figure 30 is a graph of the operating characteristics of the acquisition camera and infrared flash of the vehicle 
monitoring system; 

Figures 31 and 32 are images acquired by the system; 
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Figure 33 is a block diagram of components of the acquisition camera, and interface components for the camera 
of the image capture circuitry; "™ ra 

Figure 34 is a block diagram of communications components of nodes of the system, and the components of an 
acquisition image processing system of the system connected over the digital telecommunications network' 
Figure 35 is a diagram of the memory layout for a buffer board of the image capture circuitry- 
Figure 36 is a flow diagram illustrating software modules of the acquisition image processing system and commu- 
nications modules of the nodes; ia.iui.ommu 
Figure 37 is a block diagram of a licence plate recognition system of the vehicle monitoring system- 
Figure 38 is a flow diagram of an image acquisition procedure of the licence plate recognition system- 
Figure 39 is a flow diagram of the software modules of the licence plate recognition system- 
Figure 40 is a flow diagram of a locate plate module of the licence plate recognition system : and 
Figure 41 is a flow diagram of an optical character recognition module of the licence plate recognition system. 

[0021J A vehicle monitoring system, as shown in Figures 1 to 7, includes a camera node 2 which is mounted on a 
bridge or pylon 4 above vehicle traffic, as shown in Figures 1 to 3. The camera node 2 includes a vehicte detection 
camera 6, an image acquisition camera 8 and a node control unit 10. Both cameras 6 and 8 are monochrome^CD 
cameras, with the vehicle detection camera 6 being a wide angle video camera of medium resolution. andThe^mage 
acquisition camera being a high resolution camera. 9 
[0022] The detection camera 6 has a wide field of view 1 2 of part of a vehicle carriageway 1 6 which is to be monitored 
by the node 2. The detection camera 6 monitor* vehicies in the field of view 12 and the control unit 1 0 presses he 
.mages acquired by the detection camera 1 0 to detect and discriminate vehicles from other objects in the fieW of v lew 

b vt£r: 

2E£ ?° TkT 6 10 T V"? me Veh ' C,e 18 88 being 8 m0vln 9 whteh is <^rent '^^ other moving 
objects or the still background ,n the view 12, and determines whether the vehicle 18 constitutes an object for which 

a high resolution image thereof should be obtained by the image acquisition camera 8. The image acquisrtion camera 

B » mounted on the bndge or pylon 4 so as to have a limited field of view 20 which will include the front of a v^c e 

18 when ,t reaches a predetermined location 22 on a carriageway 16. The location 22 and the field of view 20 a e 

chosen to be near the point where moving vehicles will leave the field of view 1 2 of the detection camera 6 as shown 

, 9U , re det T nln 9 that ' he Vehfcle 1 8 represente an **« «*■* an image is to be acquired he nod" >2 
estimates the ime when me vehicle 8 will enter the field of view 20 of the acquisition camera 8, on the basis of me 
movement of the vehicle which has been monitored by the detection camera 6. The node 2 provides trigger information 
to control crcurtry associated wrth the acquisition camera 8 so as to trigger.he camera 8 at the estimated timeTZ 
resolution image f'he front of the vehicle 18 is obteln^^ 

f„ U „ C lf T *^* P ' ate detaH8, bV SubstK * uent di 9' tel etectronte P"*^ng of the image 

S JTT" t0 h ' den,ifyin9 , ,he vehicle 18 <*«™tJng the time for triggering the acquisition camera 8 the 
node 2 s able to use the images from the detection camera 6 to discriminate between vehicles on a number of char 
actensfcs, such as size, to determine those for which high resolution images are to be acquired. For exZ The 
system ,s able to dishnguish between large vehicies such as trucks and coaches, and other moving objectsZhm the 

h ££L IT TZ" a " d r ,0r S iCyCl8S - Th ° n ° d8 2 iS a,S ° «** to detelmlne <™ '"ages obte ned by 
the detection camera 6 the current speed of the vehicle 18 and whether the driver is committing any traffic or other 

rarZ eS C ri ?Jf- 9 9 ° r y*l ,a : 6 Chan9ln9 - 8ySt6m «" a,so be used 10 vehicTes 
[0024] The detection camera 6 and the control unit 10 are able to monitor all of the movino vehicles 1 8 and 22 within 
the field of view 12 whilst acquiring the images of seiected vehicles a. the iocation 22™?- T^Zl^loly 
21 , as shown ,n Figure 4, the field of view 12 of the detection camera 6 extends over all of the lanes 23 aZS%Z 
carriageway and an image acquisition camera 8 is provided for each lane 23 and 25. The node 2 is therefore abe to 
monitor the moving veh cle 18 to determine in which iane it will be when it reaches the image capture .oca fion £ and 
activates, as required, the acquisition camera 8 corresponding to that lane 23 or 25 

[0025] The control unit 10, as shown in Figure 5, includes vehicle detection circuitry 30 for processing the images 
generated by the detection camera 6 so as to provide trigger signals on a bus 32 to the hiag acqutettton carT* 8 
A selected camera 8 ,s tnggered to acquire an image In accordance with the timing infomfation determined bylhe 
detection circurtry 30 and the camera 8 provides a trigger signai on a line 36 to a flash triggering circuit 38 of a 
corresponding mfrarcd flash 40 mounted adjacent theselected acquisition camera 8. The image obtained b^ he trigger 
acquisrtion cameraeis received by an image acquisition circuit 34. The detection circuit 30 determines the liohtintens^ 

camera 8, and in turn the correct level of energy to be discharged by the flash 40 to achieve the desired level of 
exposure. The use of an IR flash is advantageous as activation is difficult to detect visually. Visible wavelengths pro- 
duced by the flash are removed by IR band pass filters. 9 p 
[0026] The vehicle monitoring system includes an acquisition image processing system 42 connected to the control 
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unit 10 for receiving and processing the images acquired by the camera 8 to extract vehicle information therefrom. The 
acquisition image processing system 42 may form part of the node 2 or be positioned remote from the node and 
connected to the control unit by a telecommunications line 44 from the acquisition circuit 34. The system 42 comprises 
a processing station 43 configured to automatically extract the required information from the image, such as licence 
5 plate details 50. 

[0027] The acquisition image processing system 42 when implemented at a remote central site, as shown in Figure 
6, includes communications controllers 55 connected to a public digital telecommunications network 45, and a central 
computer server 47 which serves a local area network (LAN) connecting computers which implement an acquisition 
image database 49, a licence plate recognition system 51 and a remote site user interface 53. The communications 

io controllers 55 are provided for each node 2 which sends images to the processing system 42. The nodes 2 each include 
an image buffer and communications controller 57 for storing images obtained by the acquisition circuit and commu- 
nicating with the communications controllers 55 of the central image processing system 42 to send the images over 
the integrated services digital network (ISDN) 45 to the central server 47. The communications controllers 55 manage 
the high speed image transfers over the ISDN 45, and handle housekeeping, error detection and correction for image 

is transfers between the nodes 2 and the central server 47. The central server 47 communicates with the controllers 55 
so the nodes 2 act as extensions of the LAN maintained by the server 47. Image processing can also be performed 
at each of the nodes 2, for example, the nodes 2 may each include a licence plate recognition system 51 which performs 
optical character recognition (OCR) on the acquired images to extract vehicle information, such as licence plate details. 
[0028] The vehicle monitoring system, as shown in Figure 7, comprises a plurality of camera nodes 2 mounted at a 

20 number of locations 52 to 60 on vehicle carriageways. The nodes 2 may be connected by telecommunications lines 
of the ISDN 45 to communicate with another and/or connected to a central control station 62, so as to compare infor- 
mation collected at each of the nodes 2. The control station 62 includes the acquisition image processing system 42. 
The nodes 2 and the control station 62 are able to monitor a vehicle's progress along the carriageways 16, . 64 using 
information collected by the nodes 2, which includes, in addition to vehicle identifying information, the date, time and 

25 location at which an image is acquired. This is particularly advantageous as the information can be used to determine 
the average speed at which a vehicle has travelled between two nodes 2. If the average speed indicates that the vehicle 
has exceeded the speed limit in travelling between the nodes, then authorities can be contacted so as to intercept the 
vehicle. Alternatively, the central station 62 issues an infringement notice to the registered owner of the vehicle. The 
station 62 and/or the nodes 2 may also contain information on stolen vehicles and the authorities are contacted when 

so a stolen vehicle is detected. Vehicle drivers negotiating long distances would be reluqtant to instantaneously exceed 
the speed limit at chosen locations, if they are aware that they will be intercepted or issued with an infringement notice 
by travelling between two locations 52 and 54 of two nodes, too quickly. The distance between the nodes would be 
relatively large and an allowable time for travel between the nodes would be established corresponding to a permitted 
average speed. The ability to monitor average speeds by the system represents a significant development which can 

35 be used to deter excessive speeding by large vehicles, such as trucks and buses, on major roads, and further enables 
detection of drivers who fail to take scheduled rest stops. 

[0029] The detection camera 6 produces video fields of 312 and 313 horizontal scan lines respectively which are 
each duplicated to produce a complete 625 line video frame. The fields are converted into 512 x 512 pixel 8 bit quan- 
tised digital images which occur at a video field period of 20 ms. The vertical resolution of the detection camera 6 is 
40 dependent on the vertical field line resolution which is approximately 300 elements, digitised into 512 pixels, for a 
maximum distance which the camera 6 can view on a horizontal roadway. The maximum distance D is given by: 

D = htan [arctan(D min /h) +<!>] (1) 

45 

where 

D = distance along road covered by camera view 
h = height of camera above road 
50 D min = distance of closest position of camera view along roadway 

<t> = lens field of view angle 

The field of view across the roadway is given by: 

55 
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(2) 



where 



W = field of view across the roadway 

w = width of the sensor 

f = lens focal length 

L = object distance from camera 



[0030] The camera 6 includes a 12 mm lens and an 8.8 mm x 6.5 mm CCD sensor to optimise vehicle imaae size 
and ma,nta,n a four lane coverage, 3.5 metres per lane, at the image acquisition points 22. An aiming Z an. 
.smear sensor is .ncluded to prevent blooming or smearing of an image by vehfcle lights. The ZreTtter of'ne 
camera permrts a infrared wavelengths up to 950 nm, which allows the detection camera 6 to receive theTnfrL ed 

camera n 6h« 122 "J? ' T*" ^ ^ i"""™" 0 " '° «»* and ™ nttor vehicTes. The de S 

camera 6 has a 440 dB gain range, and the exposure time is fixed at the field period 20 ms 

[0031] The exposure control of the detection camera 6 controls the intensity of light falling on the camera sensor so 
astoma ln talnoons»teritvideo8.gnal quality andob^napredk^ferepresemahonofavehWe. A^r e «Zu« 
of the sensor can be maintained through the appropriate match of sensor sensitivity and control of the7n» or 
power of the electromagnetic wavelength falling on the sensor, as shown with reference to equation 3 



E a (HA)T (3) 



where 



E = exposure of light on sensor i 

H = incident e.m.r. power per cm 2 (irradiance) 

A = area of pixel site in cm 2 

T = time in seconds that light or e.m.r. falls on sensor 

[0032] The time T light falls on the trigger camera is held constant at the video field rate of 20 ms This is sufficientlv 
short to freeze" the motion of the vehicle in the relatively large field of view 12 of a multMane c^iageway A sZt 
,s no mcluded in the detection camera 6 as electronic shutters or short duration exposure control ^produced S 
effects from erther image smear or blooming from sunlight reflections or vehfcle headlights, as ££2. Mmefwero 
shortened The incident light irradiance, H, required to provide sufficient exposure of a sensor pixel * dependeriTon 
the sensitive to a pabular wavelength of light. Sensor pixete also have a minimum light sensitivL ^produce a 
satisfactory s.gnaf to no.se ratio in the video signal, and a maximum light level before the Lnsor pix2b^corne sal 

fnt 2? °1 " 9M lrr ! d L anCe ** Ca " bB ima96d a 8,r *" e ex P° sure forthe is approxfrnateTloo 1 Z 
range of light irradiance which can be presented to the camera 6 during a 24 hour period can be Zed bv as much 
as 10*: . Accordingly, the exposure control system limits H sufficiently to maintain iTe eZmte r^oMhe 
sensor to prevent sensor saturation from the illumination levels typically present durino a 24 hour ™£h 
con,ro, te af1.8,o»1000autoiris,enssys«em which is designedro^ide^ 

^^^T^ ,i,terin9 °' " 9h ' " °" ' enS apertUre deCreaaeS ' - °' *£ o he exposure 
control, or the rate that H changes, is restricted as moving vehicles are located by differencing images obtained bv the 
camera 6 from a slowfy changing background image, as described hereinafter The rate ot c3e 
ensure changes ,n exposure of the sensor are not mistaken for changes in the background ^^0 ^ would ad 
versely affect detection and monitoring of vehicles. The auto iris reaction time is set t match Z «tto aS back 
ground images are subtracted from the current image. The slow rate of change also prevents *e Ws resSna too 
fast to transient ohanges in light for example, reflected off roofs of vehicles as they pass cto e to he Zem 6 The 
rate of change is restncted to 1 0 seconds for a halving or doubling of light irradiance H 

ITfL^Jl T r T*t SVStem enSUr6S ,hat ,ransient ex,reme| y bri 9 nt reflec «°ns or headlights do not saturate 
the sense pixels by limiting the exposure on the sensor to keep it below the sensors saturation level for the peak 
intense of light received in the field of view 12. The peak video level obtained from the camera oTmSore Tea 
discussed hereinafter, and used as a basis for controlling the setting of the diaphragm of the iris monllorea ' 
[0034] The sensor sensitivity Is selected in order to produce video signals which allow the subtraction of the back- 
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ground for vehicles not using headlights during dusk and dawn illumination levels. The sensor is also responsive to 
near infra-red light to maximise the signal from large vehicle side and perimeter lights, yet the response must be still 
below a threshold where blooming may occur from vehicle headlights. The lens of the camera 6 can be controlled fully 
to provide sufficient exposure for the sensor for vehicles without headlights during the dawn and dusk periods. The 
5 maximum lens aperture is held at f4 for a luminance value of about 10 cd/m 2 reflecting from the carriageway. Once 
the carriageway luminance level fall below approximately 25% of this level, vehicle segmentation, as discussed here- 
inafter, is based on vehicle headlights. Control signals representative of the illumination levels are derived from an 
illumination histogram of video signal levels for the pixels, described hereinafter. 

[0035] The control unit 10 of a camera node 2, as shown in Figure 8, includes a Motorola 68030 CPU 64 and a 
10 detection and trigger sub-system 66 connected to receive images from the detection camera 6, and an acquisition sub- 
system 68 connected to receive images from the acquisition camera 8. The sub-systems 66 and 68 include a number 
of Datacube pipelined pixel rate video processing circuit boards which are controlled by the CPU 64. The boards and 
the CPU 64 are mounted on and interlinked by a VME (Versa Module Europe) bus. The CPU 64 and the boards of the 
sub-systems 66 and 68 run a software operating system known as VxWorks, which Is a real time multi-tasking system. 
is The detection sub-system 66, the CPU 64 and controlling software form the detection circuit 30, and the acquisition 
sub-system 68, the CPU 64 and the controlling software form the acquisition circuit 34. The image buffer and commu- 
nications controller 57 can be connected to the acquisition circuit to provide access to the ISDN 45. 
[0036] The detection sub-system 66 processes the 512 x 512 pixel images of each video field obtained by the 
detection camera 6 and is designed to achieve low latency between changes in the field of view 1 2, by using pipelined 
20 processing of the image data with no intermediate storage. The data rate through the video data paths of the pipeline, 
known as MAXBUS, is 1 0 million pixels per second. Processing the video fields individually, as two consecutive frames 
of half vertical resolution, achieves a sample rate of 50 Hz and eliminates the deinteriacing latency required for full 
frame processing. 

[0037] The detection sub-system 66 includes a video digitiser board 74 which receives the fields output via the 
25 detection camera 6 and converts them into the 512 x 51 2 pixel representation. The digitiser board 74 is a Datacube 
Digimax board and produces a grayscale image representation with each pixel having a value within the 2's complement 
positive range of 0 to 127, 0 representing black and 127 representing white. The 512X512 pixels are able to produce 
a live image display as shown in Rgure 9. The image produced by the digitiser board 74 is input to a background 
differencer board 76 which, as shown in Figure 10, subtracts a background image, as shown in Figure 11 , from the 
30 current or live image to produce a preliminary difference image, shown in Figure 12. T]he difference image comprises 
a grey level of representation of the moving objects within the field of view 12. By virtue of the image subtraction the 
pixel image range forthe difference image extends from -128 to 1 27. The background differencer board 76 is a Datacube 
MaxSP board. 

[0038] The background image represents the static background viewed by the detection camera 6 and is stored in 

35 one of two f ramesto res 71 of a background image store board 70, being a Datacube Framestore board. The background 
image is continually updated by a background update board 72, which is another Datacube MaxSP board that ensures 
one of the framestores 71 holds an image correctly representative of the static background within the field of view 12 
of the detection camera 6. The update board 72 then receives the current background image from one of the framestores 
71 b and is combined with a filtered form of the preliminary difference image to produce a new background image which 

40 is outputted by the update board 72 to the other framestore 71 a. The controlling software then switches to the other 
framestore 71a for submission of the background image to the differencer board 76, and ensures the next updated 
image is submitted to the first framestore 71b. The background update board filters the preliminary difference image 
in accordance with a filter characteristic 73, as shown in Figure 10, which is held in RAM and performs a limiting function 
on the grey level pixels of the preliminary difference image so as to restrict them between a programmable range, for 

45 example -2 and +2 pixel range. The limiting function restricts the correction made to the current background image 
when it is combined with the difference image, after having been subject to a delay 74 to allow for the time taken to 
apply the limiting filter function 73. The limiting function ensures the correction made to the background image per 
frame is only slight so that transient differences, such as those produced by moving objects, are not allowed to signif- 
icantly alter the stored background image held in the image store board 70. The shape of the filter function 73 represents 

so that grey level differences added to the background image are confined to a level t for all difference levels >t and -t for 
all difference levels <-t, where t is a low threshold such as 2. The state of the background update board 72 can also 
be changed to disable update of the background image. The rate of change in the background image is set so as to 
be faster than the rate of change of scenic exposure due to variation in the lens aperture of the detection camera 6. 
The rate change governed by the limiting function is important because if the rate is too slow lighting changes can 

55 produce incorrect difference images, and if the rate is too fast then moving objects may appear in the background 
image as a blur. 

[0039] The preliminary difference image produced by the background differencer board 76 is outputted to a third 
Datacube MaxSP board, a shadow elimination board 77. The shadows produced by vehicles which appear in the 
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dtference image shown in Figure 12, pose a significant problem for the images processed to determined the type of 
vehicle. The shadows can mistakenly represent the vehicle as being larger than its actual size, and if a discrimination 
s being made between the large vehicles, such as trucks and buses, and small vehicles, such as cam and motorcycles 
then the shadow cas by a car can lead to it being dassified as a large vehicle. Therefore the shadow eliminationboard 
77 is employed to el.rn.nate all grey levels in the difference image which could represent shadows. This is done by 
defining a grey level window range 79 in RAM, as shown in Figure 10, whereby the preliminary difference image is 
processed so as to set to zero all pixels having a grey level within the window 79. The result is then used to mask the 
preliminary difference image so that the elimination board 77 outputs a shadow filtered difference image having al of 
the pixels with grey levete within the window range 79 removed. Figure 1 3 illustrates a live image with all of the pixete 
havmg a grey leve wrthin the range of the window 79 shown as green. The range defined by the window 79 is adjusted 
depending on the light conditions within the field of view 12 of the detection camera 6, as discussed hereinafter 
[0040] The shadow filtered difference image is inputted to a threshold and median filter board 78, which is a Datacube 
Snap board. The filter board 78 performs binary image processing on the difference image so as to convert the arev 
level representation of the moving objects to a binary representation, which corresponds to white or black for further 
processing by the detection sub-system 66. The fitter board 78 uses a threshold value to convert all of the pixels with 
grey levelvalueswthmtherange-^Sto+^^to pixels having values of either0or255.AccordingV.thefina^fferen* 
■mage produced by the filter board 78, when viewed by a real time display, shows the moving objects withinTe fieto 
of v,ew 12 as a co ecbon of white pbtel blobs, as illustrated in Figure 14. The blobs may correspond to parts of moving 
vehicles which reflect sunlight and. at night, may correspond to light produced by a vehicle's external lights Noise 
regions of one or more pixels in size are eliminated by the board 78 which performs binary median filtering on 3 by 3 
pixel neighbours. 9 ' 

[0041] The light conditions within the field of view 12 of the detection camera 6 are determined with reference to a 
histogram 1S0, as shown in Figure 15, of pixel grey levels produced by the CPU 64. The CPU 64 processes a window 
of the stored background image which is approximately 300 x 400 pixels every 10 seconds. The CPU 64 calculates 
the number of pixels in the window having each grey level and tabulates the results as the histogram 150 with the 
number of pixels on the vertical axis 152 and the grey level values on the horizontal axis 154. The histogram 150 can 
be displayed to provide a real time representation of the light within the field of view 12. From the grey level value 
which represents the position of the median 156. one of three lighting conditions, day, dusk, or night, can be instanU 
neously determined. Dawn ,s considered to be the same lighting condition as dusk The positions of the oeak 155 

h n ?H "J 5 ! a H mini T 158 ^ US8d ,0 detem,ine the ran9e °' the window 79 used ln «"« ^adow elimination* 
board 77. For daytime conditions, the shadow window 79 is determined as being from the values a.peak to (peak I 
median)*, where a ,s ty pi cally 0.5. For dusk conditions, the shadow window 79 is from minimum to (peak + median^ 
12. Shadow p«els of course, do not need to be eliminated during night conditions. Estimation of the shadow pixel range 
is an approximate technique which Is aided If areas of permanent shadow are in the field of view 12, such as cast by 
trees or an overpass bridge. y 

[0042] The segmented images produced by the filter board 78 are submitted to an Area Perimeter Accelerator (APAl 
J>°ard80.wh fc hfeanAPA512boa^ 

the processing of area parameters of objects in a video scene. The board 60 operates with controlling so"" 
perform analysis of the white pixel blobs within a 300 x 400 pb<el window corresponding to the window on She 
h,stogram 150 fc produced. The APA board 80 and the software perform a classification and feature extrartion precise 
in realtime on the blobs so as to facilitate the formation of clusters of blobs which correspond to a moS vehicle ^e 
APA board 80 computes features of the white pixel blobs and the features are used by the cZS^STS 

IZZ^T^ T, , Z at "t C 'T iCaSOn C ° de ' Wh8,her ,he b,0bs can be * a dust^ 

IZl' !L? , T« T Wh6ther " """P"* to ° "«Se vehicle, such as a truck or bus, or a sma^ 
veh.de such as a car. Labelling software is used to monitor movement of clusters over successive fields so asto 

TfJo 2 ^ , ° h be | asSi9ned a uni " ue label ™* which clusters are to share a label, as Ty are^n 
sidered to relate to the same vehicle. 7 

[0043] Different considerations apply in respect to whether the carriageway 1 6 is being viewed by the detection 
camera 6 at mght or during the day, and the rules and deifications used are adjusted on the basis of thedara 
provided by the histogram 150. to account for night conditions, rain and inclement «S, which res in a mo v^ 
vehide producng different corresponding pixel blobs. For example, the rules and classifies ion code needs to be ad 
justed to account for refledion produced by vehicle lights on the road during night conditions 
[0044] Once a cluster has been formed, its movement is monitored to determine its instantaneous speed and its 
position with respect to a point on the edge of the road using Kalman filter techniques. Corrections are made for 
Perspectrve as thecluaermoves towards the cameras 6 and 8. The Information obtained from monitoring the movement 
of the duster is used by the CPU 64 to predtat when the cluster will enter the field of view 20 of the acquis Jn^era 
8, and in particular when a vehicle reaches a position 22 which an image of the vehicle Is to acquired The oredtoted 
time estimate is updated for every field generated by the detection camera 6, 50 times per second. Thepredicted fime 
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is continually corrected as the CPU 64 monitors movement of a cluster until it is satisfied the cluster will enter the field 
of view within 1 0 to 20 ms. A CPU 64 predicts the time by specifying the number of scan lines which need to be scanned 
by the camera 6 before the clusters within the field of view 20. 

[0045] Performance of the control unit 10 can be monitored and controlled by peripheral devices, such as a printer 

5 94 for error and event logging, a real time status display 98, and a control workstation 1 00, which may all be connected 
to the CPU 64 and the boards of the control unit 10 directly or by a local area network 102. A display of the real time 
status display 98 is illustrated in Figure 16 which is the live image produced by the digitiser board 74 superimposed 
with cluster markings and other data. The histogram 150 is displayed at the left of the screen and the box around the 
vehicles are clusters which have been formed. The label number for each cluster is shown at the lower right hand 

10 comer of each cluster, and the estimated speed of the vehicle, obtained by monitoring the cluster, is displayed directly 
below the label number. The large box around the vehicles represents the processing window, on which the clustering, 
labelling and tracking software operate, in addition to the histogram software. The line across the window is an acqui- 
sition line which corresponds to the position 22 at which high resolution images are to be acquired by the acquisition 
camera 8. A diagnostic graphics board 82, which is a Oatacube Maxgraph board, is used to queue and configure 

is graphic images for the real time status display 98. 

[0046] The image processing performed by the CPU 64 and the APA board 80 for vehicle classification is handled 
by feature extraction, clustering, labelling and tracking software. The operation of the software is largely controlled by 
parameter variables, which may be altered via an interactive shell of the software or by remote procedure calls from 
a graphical interactive command tool running under Xwindows on the control workstation 100. 

20 [0047] The APA board 80 reduces the binary image pixels into a stream of feature vectors representing the blobs, 
or regions, in the images. Only a small sub-set of the features which can be computed by the APA are required, being 
the area, perimeter and bounding box for each blob, or region. A region is represented by raw data of 16 bytes and 
for a field of view 12 which includes 20 regions, the data rate is 16 kbytes/s which is less then 0.2% of the data rate 
for binary images, and is reasonable for software processing by the CPU 64. 

25 [0048] The raw seed parameters are read from the APA hardware by the APATask 1 70, as shown in Figure 17. A 
time stamp is given to each blob, and some initial screening is performed, where regions such as "black triangles" 
described hereinafter, are located and removed. Time stamping, inter alia, allows any latency in the system to be 
measured and compensated for. The seeds which correspond to white blobs within certain area constraints are passed 
via a VxWorks message pipe to the seedTask 172. The seedTask unpacks the raw seed parameters, or structures, 

30 and performs classification of regions based on each regions height to width ratio, "circularity", area and "coverage", 
as described hereinafter. Unwanted regions such as headlight and road reflections are removed and then each clas- 
sified region is passed via another message pipe to the clusterTask 1 74. 

[0049] The clustering task is divided into five subsections 176, region classification, region extension, clustering, 
region unextension and cluster classification. Once the regions have been clustered into clusters which have been 

35 classified as corresponding to separate vehicles, the coordinates of the clusters are passed onto a label task 1 78, once 
again by a message pipe. The label task monitors each cluster over a given period of time and if a cluster appears in 
roughly the same place as did a cluster from a previous video frame, then the label task considers them to be the same 
cluster. In this case, the new cluster inherits the label from the previous cluster. Otherwise if no match can be made, 
the new cluster is given a new label. The cluster's coordinates, along with its label, is then passed via another message 

40 pipe to a trajectory task 1 80. The trajectory task 180 determines the time to trigger the acquisition camera 8 for clusters 
of a selected class, such as targe vehicles. The put cluster box task 182, remove cluster box task 1 84, put label task 
186, remove label task 188 and the histogram task 190 are tasks used to generate graphics overlaid on the video 
image, as shown in Figure 1 6, for diagnostic purposes. 

[0050] The blob shape analysis performed by the APATask 170 and seedTask 172 is not extensive during daytime 
is segmentation, as all blobs are considered valid. However, during dusk and night time segmentation, blobs can occur 
due to vehicle headlight reflection, and if these blobs are clustered in with true vehicle blobs, then the front-of -vehicle 
coordinates, which are taken from the bottom of the cluster box, will be incorrect. In order to correctly locate each 
cluster box at the front of each vehicle, blobs which are recognised as being due to headlight reflections are identified 
and removed before blobs are clustered. Other problem blobs are those which correspond to road lane markers. These 
so appear when the mount for the detection camera 6 shakes. During camera shake, the incoming video image no longer 
precisely corresponds to the stored static background image, and therefore the result from the background image 
subtraction is that the road markers appear to have moved. Again, the blobs that result from camera shake are identified 
and filtered out before clustering commences. A further problem is "black triangles". The APA board 80 possesses a 
hardware fault which causes the polarity of blobs to be specified incorrectly. If a black region finishes at the right hand 
55 side of the pixel processing window, it can be inadvertently labelled as a white region by the APA board 80. These 
white regions can then become candidates for clustering unless filtered out by the seedTask 172. Typically, when a 
lane marker 190 appears on the right side of the pixel processing window 192, as shown in Figure 18, it produces a 
black triangular blob 194, a "black triangle", which is inadvertently represented by white pixels, in the top right hand 
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corner. The triangular blob 1 94 is identified and removed. A convenient side effect of the polarity fault is that the road 
lane line-marker 190, which usually must be identified and removed by other shape characteristics, is labelled by the 
A PA board 80 as black, and is therefore automatically filtered out. 
[0051] Regions are classified into one of the following types: 

(i) Headlight reflections; 

(ii) Road artefacts; such as road lane markers, which appear due to camera shake, 

(iii) Lights; and 

(iv) Other; during daytime segmentation most of the regions that are not classified as road artefacts are classified 
"other". 

[0052] During day and dusk conditions, illuminated headlights do not appear segmented from other segmented parts 
of a moving vehicle, and so lights are not classified. At night, however, circular regions are classified as either "headlight" 
or "small-light", depending on the area and position within the field of view 1 2. Distant headlight pairs which are typically 
segmented from the background image as a single joined region, are classified as "joined headlights". To obtain correct 
initial clusters, distant joined headlights need to be distinguished from the small perimeter lights of large vehicles. 
[0053] The main shape measure that is used during dusk and night time processing is "circularity". This is a measure 
which considers how close each blob is to the shape of the circle by comparing the blob's area to its perimeter In the 
case of a circle: 

Area = jcr 2 ^ 

Perimeter =2nr (5) 

[0054] The radius term can be eliminated, since it is only relevant for circles, by squaring the perimeter equation and 
taking the quotient of the two terms. For a circle, this produces a constant: 



Area nr 2 1 



(Perimeter)^ {2nr)' 



(6) 



[0055] To make a circularity measurement equal to 1 for a circle, equation 6 is simply multiplied by the inverse of the 
constant. This provides a circularity measure which can be applied to blobs whereby a circular blob will have a meas- 
urement value of 1 , as follows: 

Circularity = 4?tArea . 1.0 m 
(Perimeter) 2 K ' 

[0056] For a square blob of unit area, Area = 1 , Perimeter = 4, the circularity measures is as follows: 

Circularity -^r-J- 0.785 {8) 

[0057] For an equilateral triangle with sides of unit length, Area = V3/4, Perimeter = 3, the circularity measures is as 
follows: 

Circularity »4 B . | • (9) 

[0058] A further measurement employed, that is particularly useful in detecting road land/line markings, is "coverage". 
Coverage is the measured ratio between the area of a blob to the area of its bounding box. The bounding box 200 as 
shown in Figure 1 9, is aligned with the APA board coordinate axes, which are the axes of the APA processing window 
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The APA axes are not necessarily aligned with the major axis of the blob itself. For instance, a rectangular blob 202 
which is aligned with the APA coordinate axes would have a high coverage value, whereas a rectangular biob 204 
which is not aligned with the axes may have a medium coverage value. A concave shape 206 would produced a 
medium coverage value, and a line 208, diagonal to the APA coordinate axis 201 would produce a low coverage value. 
5 Road lane markings can be simply detected because they have a low coverage value. If the lane markings are not 
diagonal, but vertical, then the measure is insufficient and in such cases a measure of the ratio of the blob's major axis 
length to it's minor axis length can be used instead. 

[0059] During night time segmentation the coagulated blobs of joined headlights are identified by their height to width 
ratio as they tend to be twice the expected area of one headlight. Joined headlights need to be detected so that a 
10 headlight count maintained for each cluster is correct. 

[0060] Headlight reflections appear as large elongated blobs, and are detected initially on the basis of their size and 
characteristic shape. The blobs are confirmed as relating to headlight reflections by extending the blobs vertically to 
determine whether they extend from a headlight region. 

[0061] As the vehicle monitoring system is capable of continuous automatic operation, clustering of regions takes 
is into account different lighting conditions. The technique of static background subtraction, described previously, seg- 
ments moving objects from the video image obtained by the detection camera 6, but the regions that result from the 
segmentation process depend on the ambient lighting conditions at the time of day. During daytime segmentation, 
large regions typically result, whereas during night time only headlights and the smaller sidelights on trucks are seg- 
mented. During dusk, lit headlights do not appear segmented from the other visible parts of moving vehicles, however, 
20 reflections upon the surface of the road caused by the headlights need to be removed, as discussed above. 

[0062] The clustering process operates on the segmented regions or blobs and each vehicle is typically segmented 
into several separate regions, as shown in Figure 12. For instance, a car will often appear split by its windscreen into 
a roof-region and a bonnet-region. Large vehicles typically segment into more regions. The cluster task groups these 
regions into "logical vehicles' 1 so that they can be tracked. Distant vehicles tend to be segmented together into one 
25 region due to vehicle occlusion at the image horizon. While the segmented regions at this distance can be tracked, 
they cannot be reliably clustered into separate vehicles. Emphasis is placed on correctly clustering lower regions that 
are closer to the acquisition line 22, and consequently the clustering process scans from lower regions to higher regions 
in each image. 

[0063] If two vehicles are segmented into the same region, then they will be clustered together. The cluster task does 

30 not separate vehicles that have been segmented together into a single region. The coordinates of each cluster are 
sent to the label task 178 which matches and separates clusters over consecutive video fields. The cluster task and 
the label task classify clusters on the basis of classification data. The coordinates passed to the trajectory task 1 80 
correspond to an estimation as to the front of the vehicle, at the road surface level. Cluster information on all vehicles 
is provided to the trajectory task, which tracks the clusters and selects for which vehicles images are to be acquired. 

35 [0064] Optimal clustering is achieved as a middle point between "over clustering" and "under clustering". At the over 
clustering extreme, all regions are clustered into one single cluster and then only the lowest vehicle in the cluster is 
tracked. This is because the lowest point of each cluster is passed by the label task to the trajectory task. The classi- 
fication of the cluster, which is based on its height and width will be incorrect. At the under clustering extreme, if no 
regions are clustered together, that is each region obtains its own unique cluster and label, then the trajectory task is 

40 over-burdened in an attempt to track every region, vehicle classification will fail in a number of instances, and images 
will be inadvertently acquired and missed. For the purposes of vehicle image acquisition, it is better to mistake a vehicle- 
roof for a vehicle-front and begin to track it than it is to mistake a vehicle-front for vehicle-roof and so, by adding it to 
the back of another cluster, not track the vehicle-front. Therefore the cluster task has been written to use an optimal 
middle point which lies on the side of under clustering rather than over clustering. 

45 [0065] The cluster task performs clustering essentially by extending the boundary of each segmented region by a 
certain amount, and then joining any regions that overtap. Regions that overlap are "clustered". The cluster task, how- 
ever, determines correctly the amount of extension which should be applied to each region. During daytime segmen- 
tation, very little region extension is required, yet during night time, the segmentation process produces small sparse 
regions that require large amounts of extension in order to achieve overlap. 

so [0066] An important aspect in the construction of a cluster is that the bottom region of each cluster should be the 
front of a vehicle. Invalid regions, such as regions due to headlight reflections, must not be clustered, and are thus not 
extended. After every valid vehicle region in the image is extended by a certain amount, the clustering process begins 
with the lowest region in the image. The lowest is considered first which is the region most likely to cause triggering of 
the acquisition camera 8. 

55 [0067] The coordinates of the lowest region are used to initialise a cluster structure. Then all extended regions above 
the initial region are tested for overlap, if any region does not overlap with the coordinates of the cluster, then the 
cluster coordinates are updated to include the region and the region is marked as clustered. Whenever a new region 
is added to a cluster, all remaining un clustered regions become possible cluster candidates again. Thus the list of 
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regions is traversed again from the bottom of the image. Although the regions in the list which have already been 
marked as clustered can be skipped, this is considered sub-optimal. Once the entire list of regions have been traversed 
without any overlap detected, the next cluster is begun with the lowest remaining region. The clustering process con- 
tinues in this manner until no regions are left unclustered. The list of clusters are then unextended and passed to the 
label task. 

[0068] In performing region extension, regions are extended by a variable amount in the vertical direction, but ex- 
tended by a standard amount in the horizontal direction, with reference to the APA coordinate axis. Horizontal extension 
is unnecessary during daytime segmentation, as a vehicle blobs tend to be connected across the full width of the 
vehicle. It is in the vertical direction that blobs due to the same vehicle appear disconnected. For example, two blobs 
that typically represent a car might be due to its bonnet and its roof. These two blobs will stretch over the full width of 
the vehicle, and appear one above the other. Furthermore, so long as one blob for each vehicle stretches the full width, 
the cluster coordinates will be wide enough to incorporate any blobs that might otherwise need horizontal extension 
to be clustered together. The full width blob provides the extension. With reference to the example illustrated in Figure 
20, the region 210 becomes added to the region 212 on the right, from which the cluster 214 is begun, only because 
the full width region 21 6 above was added to the region 212 to form the cluster 21 4. If the region list was not researched 
from the beginning of the list continuously, the overlap of the previously tested region 21 0 would not have been found. 
It is for this reason that the clustering task, as discussed above, reconsiders all unclustered regions after adding a 
region. 

[0069J The cluster task is able to perform one of two extension methods. The first method takes the vertical or Y 
coordinate of the region as an input to a look-up table that specifies the amount of extension to be applied. The amount 
of the extension, and hence the degree of clustering, is then modified according to lighting conditions. As the outside 
light level decreases, and regions reduce in size, the amount of extension applied to regions can be gradually increased. 
Furthermore, the perspective in the image can be compensated for by adjusting the values stored in the look-up table 
accordingly, i.e. distant regions high in the camera image can be extended less than near regions which are low in the 
image. An example of the extension values stored in the look-up table is illustrated by the graph 21 8 shown in Figure 
21 of extension amount v. Y coordinates. All extension amounts are provided in pixel numbers. The second extension 
method extends each region by an amount proportional to its width. The method is largely based on an observation 
of the shapes of regions obtained during daytime segmentation. Small regions, which are typically far away, are min- 
imally extended, large vehicle body regions, which are typically close, square and occunone per vehicle, are minimally 
extended, and wide short regions, which are often vehicle fronts, are greatly extended. Essentially, as illustrated in 
Figure 22, this results in every region boundary 220 and 222 being approximately square. In Figure 22, the boundaries 
220 and 222 of both regions 224 and 226 have been extended vertically to equal at least their width. Therefore the 
wide short region 224 has been extended a great deal more than the large square region 226. Region 224 would be 
a vehicle front portion disposed under the vehicle body region 226. Therefore, the two regions 224 and 226 can be 
matched without too much extension. If the large region 226 is over extended, then it may overlap with a succeeding 
vehicle front. In the preferred embodiment, this method is only employed during daytime segmentation as night time 
processing requires a large amount of region extension, although it is envisaged the extension factor used in the 
extension calculation can be enlarged for night time use. 

[0070] During night time clustering all of the regions to be clustered are essentially small circles, and a truck cluster, 
for example, is constructed by considering the possibility of whether each light could feasibly fit into a stored truck 
template. For the first region in a cluster, to fit within the template, there is a maximum distance of light separation 
which cannot be exceeded. 

[0071 J Overlap of regions is detected by comparing the coordinates of regions and clusters, wherein the top-left (x 1t 
y t ) and bottom-right (x 2 ,y 2 ) coordinates for both regions and clusters are known. For the image plane coordinates, x 
increases from left to right and y increases from top to bottom. Considering first the horizontal, x coordinate, overlap 
for the regions R 1( R 2 , R 3 , R 4 , R 5 and *6 illustrated in Figure 23 the test for overlap with the cluster C n is: 



R n( x i)<C n (x 2 ) (10) 



C n(Xi)<R n (x 2 ) (11) 

[0072] If both of the two equations are true, then there is overlap in the horizontal direction. Therefore, horizontal 
overlap is true for fy. ^4 and R 5 but region fails the test as equation 1 0 is not true and region Rg fails the test 
as equation 8 is not true. A similar test is performed in the vertical direction as follows: 
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Rn(y 2 )>c n (y 1 ) 



(12)- 



[0073] There is no need to perform the complimentary test for R n (yt) because the regions are outputted from the 
5 APA board 80 in order from top to bottom and as the cluster task processes all regions in a list from the bottom up, the 
complimentary test, C n (y 2 ) > R n (yi). is unnecessary as it will always be true. 

[0074] Clustering during day lighting conditions is based on the overlap test discussed above, yet during dusk and 
night conditions clustering involves consideration of additional rules, primarily due to the increased ambiguity and 
greater separation between regions of the same vehicle. Certain regions should also never be clustered, such as 
10 headlight reflections and noise from background image areas due to vibration of the detection camera 6 discussed 
previously. Clustering therefore also involves consideration of a series of rules based on the various region classifica- 
tions discussed previously. The rules include: 

(i) An extended region must spatially overlap a cluster to be added to that cluster. 
'5 (ii) If a region overlaps more than one cluster, then it is added to the lowest cluster. 

(Hi) A region to be clustered cannot already have been added to the cluster. 

(iv) A "joined headlights" region cannot be added to an existing cluster. Regions of this type can only initiate a 
cluster. 

(v) Only a predetermined number of "headlight" regions can be added to a cluster, the predetermined number 
20 being a system parameter which can be adjusted from the user interface. 

(vi) As many "other" and "small light" regions as is spatially allowed can be added to a cluster. 

(vii) A region which touches or includes part of the top of the processing window can initiate a cluster but cannot 
be added to a cluster. 

(viii) A further "headlight" region to be added to a cluster must be horizontally aligned with another "headlight" 
25 region in that cluster, which is determined on the basis of the difference between the regions lower Y coordinates. 

(ix) "Reflection" and "road artefact" regions are not added to any cluster 

[0075] For monitoring a roadway, clusters are classified into one of three classes: car, ute (a small flat-bed utility 
truck) or truck. Therefore all large vehicles, such as buses and articulated vehicles, are classified as a truck. Cluster 

30 classification is based on the height and width of each cluster box, and the number of; lights within the cluster during 
night conditions. The height and width data for each classification is modified via procedure calls to the histogram task 
1 90 as the lighting conditions change from day to dusk and to night, etc. The cluster width is as important as the cluster 
height because, for example, a large four wheel drive vehicle towing a trailer might produce a cluster which exceeds 
the truck height threshold but is unlikely to be as wide as a truck or bus. A histogram of cluster heights and widths of 

35 motor vehicles includes distinct peaks which correspond to various vehicle classes, and is used to set the stored 
classification thresholds automatically. The height and width histogram is shown in the display of Figure 16. For ex- 
ample, a cluster is classified as a truck if one of the following conditions is true: 

(i) The cluster height and width exceed the truck threshold. 
4 ° (ii) The lighting condition is night and the cluster exceeds the truck width threshold. 

(Hi) The lighting condition is night and the number of small lights in the cluster exceeds the small light truck threshold, 
(iv) The cluster height is within a predetermined range of the truck height threshold and the number of small light 
regions in the cluster exceeds the truck small light threshold. 

45 [0076] As ambient lighting drops, the size of the truck clusters are reduced, and consequently the height and width 
thresholds decrease, depending on the lighting conditions, as determined by the histogram task 1 90. The classification 
for each cluster is stored in a clustered data structure, together with the cluster's coordinates and time stamp. The 
clustered data is then passed to the label task 178. 

[0077] The label task 1 78 assigns a label to each unique cluster and tracks clusters over time by matching an array 
so of previously seen clusters to each subsequent video field of clusters. If a cluster appears in roughly the same place 
as a cluster from a previous field, then the label task 1 78 considers them to be the same cluster. Where a match can 
be made, the new cluster inherits the unique label of the previously seen cluster. If a cluster cannot be matched, then 
a new label is created for that cluster. Clusters may disappear for a few fields, and it is an objective of the label task 
178 to determine whether a cluster is indeed new or whether it has just appeared again after a period of absence. 
55 [0078] The matching of dusters is based on location. Cluster size can be used as an extra match parameter but the 
current location heuristic has been found sufficient. It can be assumed the clusters will not move very far from their 
position in the previous field, and if a cluster moves so far that its boundary coordinates in the present frame do not 
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overlap with its boundary coordinates from the previous frame, then the previous label will not be transferred Clusters 
can split and join, both vertically and horizontally, as they are tracked from field to field. Two labelling methods have 
been developed, with the second being the preferred method which is presently used. 

[0079] The first labelling method involves two reciprocal tests which are used to determine whether a new cluster 
should inherit an old cluster's label. The first test is to determine whether the centre of a new cluster 230 lies within 
the boundary of any clusters 232 and 234, as shown in Figure 24, on a list of previously seen clusters, called the label 
list. For the cluster 230, the test fails, but for the new clusters 236 and 238 their centres fall within the older cluster 240 
so the lowest new cluster 238 inherits the label of the old cluster 240, and the upper new cluster 236 is assigned a 
new label. The second test, which is executed when the first test fails, determines whether the centres of any of the 
clusters on the label list lie within the boundaries of the clusters from the current video field. Therefore as the centres 
of the old clusters 232 and 234 fall within the boundaries of the new cluster 230, a match is detected and the new 
cluster 230 inherits the label of the lower old cluster 234. Applying the second test to the new clusters 236 and 238 
results in failure as the centre of the old cluster 240 does not lie within any of the new clusters 236 and 238 and 
therefore applying this test to these clusters would result in the new clusters 236 and 238 both being assigned new 
labels. 

[0080] The second labelling method is based on the clustering overlap technique described previously Essentially 
the bounding box of each cluster from the current field is tested for an overlap with clusters in the cluster list The 
cluster list is search from bottom to top, in a similar manner to the search method described for detecting overlapping 
regions. In this way, if two clusters merge into a single cluster, then the first overlap found will be an overlap with the 
lower cluster. Once a match is found, the search is terminated, and the label which is transferred is marked as applied 
to a new cluster. Therefore a label cannot be transferred twice within one search of a new video frame. The second 
method is preferred as it requires half the number of tests as the first method, and a cluster can move further between 
successive frames yet still inherit its label. In the first method, where centres are matched to the boundaries the 
maximum displacement allowed between fields is half the width (or height) of the clusters, whereas in the second 
method, where boundaries are checked for overlap, the maximum displacement is the entire width (or height) of the 
cluster. Therefore the second method allows a cluster to move twice the distance of the first method. 
[0081] As clusters travel successive fields in time, they tend to split or join, and if a cluster splits, then the label is 
transferred to the lower of the two clusters, and the upper cluster, which would typically be another vehicle behind the 
lower cluster, is provided with a new label. Alternatively, if two clusters join, then the ofa lower cluster's label is trans- 
ferred to the new combined cluster, and the other cluster's label is allowed to expire. The label of the lower of two 
clusters is transferred after a split or join because the lowest cluster is most likely to include the front of a vehicle and 
is therefore given priority with regard to maintaining cluster labels. 

[0082] A record of the bounding box coordinates is maintained for each cluster in the duster list together with its 
label, the labels age, and when the cluster was last seen. Whenever a label is inherited, its age increases and its last 
scene value is reset. If a label is not transferred in the course of one field, its last scene value is incremented A label 
is removed from the cluster list if its last scene value exceeds a label tenure threshold. Cluster labels, coordinates and 
classifications are passed to the trajectory task 180. 

[0083] The trajectory task 1 80 uses the received cluster data to track the clusters over successive video fields The 
coordinates used for tracking a cluster box are the coordinates of the centre of the base of the box, and the coordinate 
system for the roadway 1 6 which is adopted is illustrated in Figure 25. The datum 300 of the roadway coordinate system 
is an arbitrary point on the roadway, which has been chosen as the centre of the left hand fog line underneath the edge 
of an overpass bridge holding the cameras 6 and 8. Vehicles 302 travel in the positive Y axis direction on the roadway 
16, starting at a negative value in the distance. The trajectory of a cluster box in image plane coordinates (x, y.) as 
shown inthe graph of Figure 26 is notlinearwith time due to the effect of perspective. Therefore a camera transformata 
is applied so as to convert image plane coordinates to real world 3-D coordinates. In matrix form, the overall camera 
transformation is as follows: 
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where 



a, X-axis scaling factor in pixels/mm (intrinsic) 
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oty Y-axis scaling factor in pixels/mm (intrinsic) 
Xq image plane offset in pixels (intrinsic) 
Y 0 image plane offset in pixels (intrinsic) 
f focal length (intrinsic) 

°T CAM detection camera 6 position in world coordinates (extrinsic) 

[0084] The intrinsic parameters are innate characteristics of the camera and sensor, while the extrinsic parameters 
are characteristics only of the position and orientation of the camera. The principle point of the image plane is the 
intersection of the optica! axis and that plane, at coordinates (Xq.Yo). Equation 13 can be written as: 
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where C is the camera calibration matrix, a 3 x 4 homogeneous transform which performs scaling, translation and 
perspective correction. The image plane coordinates are then expressed in terms of homogeneous coordinates as: 



(15) 



25 



2 



(16) 



:~7 



[0085] The general perspective transform maps a ray in three dimensional space to a point on the image plane. For 
30 vehicle coordinates in the image plane as seen by the detection camera 6, a unique three dimensional location of the 
vehicle cannot be determined so the bottom of a cluster box received from the label task is considered to be on the 
roadway, i.e. 2 = 0, and therefore the box can be tracked with reference to the roadway x and y coordinates. The 
equations 14, 15 and 16, given the image plane coordinates and z, can be solved simultaneously for the roadway 
coordinates x and y to specify the position of a vehicle. The equations have been solved using the computer algebra r 
35 package MAPLE, and the solution, in C notation, is as follows: 

den = (-Xi # C31*C22+XrC32*C21+(Yi*C21-C21)*C12+(-YrC32+C22)*C1 1); 



y = -(-XrC3rC24+Xi # C34*C21+(YrC21-C21)*C14 + 
(Xi*C33*C21 -Xi*C31 *C23)*2+(Yi*C31 -C21 *Z*C1 3+ 
(-Yi*C34+C24+(-Yi*C33+C23)*2)*C1 1 )/ den; 



X = (-C24*XrC32+C22*Xi*C34+(Yi*C32-C22)*C14+ 
(C22*XI*C33-C23*XrC32)*2+ 
(Yi*C32-C22)*2*C13+(-Yi*C344C24+(-YrC33+C23)-2)*C12) / den; 



[0086] The solution explicitly includes height above the roadway, z, which can be set at zero for daytime operation 
55 or some marginal distance above the roadway, whereas at night, the bottom of the cluster box generally corresponds 
to the height of the headlights above the road, and therefore 2 is set to a notional headlight height. Figure 27 illustrates 
a graph of the same vehicle trajectory as in Figure 26, after the trajectory has been mapped to the roadway coordinates 
x and y. The trajectory illustrates the vehicle is moving at a constant speed, and in the left hand lane. 
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[00871 The time at which the vehicle 302 will reach the acquisition line 22, and the future location of the vehicle 302 
need to be predicted, due to latency in the system. Considerable latency exists between a trigger 
acqu.srt.on v,athe acqu.srt.on camera 8, and additional latency is caused by pixel transfer, image process Z'Z 
delay and software processing delay. The information obtained on the basis of the images required by the de e^ on 
camera 6 prov.de a delayed representation of the actual vehicle position, and therefore it is necessaTest^he 
future position and speed of the vehicle 302. w.«*«»ry estimate tne 

[0088] The position estimates of a vehicle obtained by the inverse perspective discussed above are quite noisv due 
to quanhsat, on effects, particularly when vehicles are in the distance, therefore simple differencing cann^be Zd to 
esbmate velooty of a vehicle and therefore the software uses a Kalman filter to reconstruct the veNde* laterauLd 
ongrtud nal position and velocity states, based on the noisy observations of the vehicle p£ n Z veh fee Se £ 
each of the longitudinal and lateral axes comprises position a, and speed o> of the vehicle, represented as follows 

X = [co6] T (17) 
[0089] In space state form, assuming constant velocity motion, the vehicle dynamics are 



X = *x (18) 



^ = C * (19) 



where Y is the observable output of the system, being the vehicle's lateral or longitudinal position <t is the state- 
trans,.,on matnx, and C is the observation matrix. For constant velocity motion the matrices are as follows 



« 




(20) 



c = [ioj (21) 



T£ I- SamP ' in9 intena1, bei " 9 eqUa ' t0 ' he Vid6 ° ,ie,d in,e ' Va, which fe 20 ms - The Kalman "Iter equations 



for one axis are 

K = OPC T (CPC T +R 2 )- 1 (22) 

>U<i>X + K(y-CX) (23) 
P = OP<I> T + R 1 l 2 . KC p 0 T 

[0090] The niter Is predictive, and X is the predictive value of the vehicle state for the next sample interval K is a 
ga n, P is the error co-vanance matrix, and l 2 is a 2 x 2 identity matrix. R 1 and R 2 are input and output 
estimates, and are used to adjust the dynamics of the filter vanance 
[0091] The Kalman filter equations 22, 23 and 24 are complex and time consuming to execute in matrix form and 
the computer algebra package MAPLE was used to reduce the equations to scalar form, as follows, Sa 

r compute the filter gain 7 

den = kp->p11 + *R2; 

k1 = (kp->p11 + T • kp->p12) / den 

k2=kp->p12/den; 
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/• update the state vector */ 

X 1 = kp->xl + T ' kp->x2 +kl ' ('y - kp->x1); 

x2 = kp->x2 + k2 • Cy - kp->xl); 

kp->x1 =x1; 
kp->x2 = x2; 

/•update the covar matrix (symmetric so keep only 3 elements)-/ . 
p11=-R1 + kp->p11 + 2.0-T-kp->p12 + T-T-kp->p22-k1 kp-> P 11-k1 kp-> P 12 T, 

pi 2 = kp->p12 + T ' kp->p22 - k1 * kp->p12; 
P 22 = *R1 + kp->p22 - k2 * kp->p12; 

kp->p11 =p11; 
kp->p12 = p12; 
kp->p22 = p22; 

ro0921 The estimated values for the state of the vehicle and error covariance for the filter are calculated using the 
Sons and adored in a data structure -ftp. Optimal values for R, and R, are determined » 
nius^tes qZhs which can be plotted from the estimated values for one axis, being the estimated position and esb- 
matld soee7of me vehicle, and the estimated error associated with the camera filter calculations, as each v,deo field 
SKSS acquisition time is ca.cu.ated by using the estimated vehicle state data^As the posrtion ,22 
at wTh^s^nTo o ecu, Ys known, the estimated acquisition time is emulated by taking the d.fference between 
£ iZ position and the acquisition position, and dividing the result by the estimated 
WhenTe estimated acquisition time falls below a value whteh indicates acquis.tion ,s to occur w,th n me time , of the 
^ext vidTfield then the estimated time information is provided to a trigger board 84. The £jm*d veh.le state 
^ordinate for the x direction indicates which camera 8 of a multi-lane carnageway is to be tnggered. 
STJSJS C of equation 14 may be calibrated using road markers or preferably telescope stakes 

KUZedatP^ 

datum 3^0 to obtain the x, y and z coordinates for different positions on the stakes, and then removed. Equation 14 
can be expanded as follows: , 



C„x + C 12 y + C 13 z + C u - C 31 X'x - CjjX'y - C^Xz - C^X' 



(25) 



C 21 x + C^y + C 23 z - C 24 - C 31 Y'x - C^Y'y - C M Y'z - C^Y' = 0 



(26) 



which relate an image plane coordinate (X'.Y) to a real world coordinate (x,y,z). For n observations this can be 
expressed in matrix form as follows: 



x, y, z, 1 0 0 0 0 -X',*, -X',y, -X',2, 
0 0 0 0 x, y, z, 1 -V,x, -Y',y, -Y',z, 



x, y. z„ n 0 0 0 0 -X' s x„ -X'.y,, -X',z B 
' 0 0 0 x B y B z B n -V.*, -Y'.y,, -Y' 0 z„ 



'11 



'12 



c, 3 



X', 
Y', 



X'. 

rJ 



(27) 



100941 The equations are homogeneous and therefore the overall scaling of the C matnx s s,mply chose so that 
C 1 and Ms parameter is not identified. Equation 27 has 11 unknowns and for a solution requires at least 5.5 
oteeUfions. be.rJpa.ra of (X.Y) and (x.y.z). The system of equations is generally over de.erm.ned, and a .east 
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426, as shown in Figure 39. The locate plate module 420, as shown in Figure 40, begins at step 430 by preparing the 
1280 x 1024 pixel image for processing as a number of pixel windows for the Pixar co-processor 402. At step 432, 
the system 51 attempts to detect an edge of a character size object, and when detected the object's location is deter- 
mined at step 434. An object assembler is used at step 436 to group adjacent objects together, and the groups are 
5 processed by a plate classifier 438 to determine whether the object groups could constitute a licence plate. If an object 
group is classed as a plate according to a plate template, a bounding box is formed, and its coordinates returned to 
the glyph extraction module 422. The glyph extraction module 422 processes each bounding box to binarise and extract 
individual characters in a bounding box and then pass the "glyphs", i.e. licence plate letters and numbers, to the OCR 
module 424. The OCR module 424, as shown in Figure 41 begins at step 428 by building a typological graphical 

10 representation of a glyph from the glyph bitmap provided by the glyph extraction module 422 for each glyph. The 
graphical representation is analysed at step 440 so as to detect any characteristic features, such as holes, arcs and 
vertical and horizontal lines. From the results of step 440 an 81 bit string representing the characteristic features of 
the glyph is created at step 442. A bayesian statistical analysis is then performed at step 444 on the feature string to 
try and match the features against a set of previously determined features characteristic of known ASCII characters. 

is The ASCII value of the match with the highest probably of being correct is returned to the plate recognition module 426. 
[0115] The plate recognition module 426 determines whether the glyphs in a bounding box constitute a valid licence 
plate. The module 426 effectively controls the other image processing modules as it has the ability to override the 
results of the OCR module 424 or to force the glyph extraction module 422 to use a bounding box other than that found 
by the locate module 420. The majority of vehicle licence plates in Australia have six characters and fall into one of 

20 two classes, Federal plates or non-Federal plates. Federal plates comprise two alphabetic characters, two digits and 
two alphabetic characters, whereas non-Federal plates comprise three alphabetic characters and are followed by three 
digits. The plate recognition module 426 is able to determine whether a valid licence plate has been found on the basis 
of this information, and other information, such as the spacing of characters and the specific characteristic alphanumeric 
sequences used by the non-Federal plates. The OCR module, for example, may not be able to distinguish between 

25 capital B and 8, and for many plate fonts, there is no difference between a 0 and O or a 1 and an I. Therefore the plate 
recognition module 426 may need to override the results obtained by the OCR module 424. The plate recognition 
module 426 is also able to instruct the glyph extraction module 424 to process an altered bounding box if the module 
426 determines that there may be additional glyphs to the left or right of an original bounding box returned by the locate 
module 420. The licence plate details obtained by the plate recognition module 426 are stored on DatabaseQ 415 of 

so the server 47, and archived on the optical disk 41 0. The optical disk 41 0 also archives image files which the system 
51 is unable to process when received. 

[0116] The database on the optical disc 410 stores for each processed image as does DatabaseQ 415, data con- 
cerning the position, size and characters of the numberplate located in the image, and other details such as time and 
date of acquisition. It is also structured with data pointers which facilitate access to the stored data by the workstation 
35 400. The workstation 400 includes graphical user interface software which enables an operator to review the results 
of the procedures 412 and 414, and perform further optical character recognition on numberplate regions, as selected. 
Any further OCR processing performed on a plate region selected by the operator of the workstation 400 is normally 
used to analyse the performance of the procedures 412 and 414 and not to alter the integrity of the data held in the 
optical disc 41 0. 

40 [01 1 7] The image data stored on database 421 is processed by matching software which looks for matches amongst 
the licence plate details fields of the image data so as to locate occurrences of detection of the same licence plate at 
different remote sites or nodes 2. Once a match has been located, the acquisition time fields can be used to determine 
whether speed or time violations have occurred in travel between remote sites 2, as distance between the sites 2 is 
known. The matching software is run on a Sun Microsystems workstation 450 connected to the LAN 41 9, or alternatively, 

45 the matching software is run on a system of a road traffic authority, with the image data being sent by the central server 
47 over the ISDN 45 to the road traffic authority. The road traffic authority is able to communicate with the central server 
47 via the ISDN 45 to obtain archived images, as required. 

[01 1 8] To avoid sending all images to the central server 47, a large number of which may not be of interest, images 
can be archived at the nodes 2, and licence plate details extracted at the remote nodes 2 by respective licence plate 
so recognition systems 51 connected directly to the BIT3 repeater cards 89 of a node's acquisition sub-system 68. The 
server 47 then only receives the extracted licence plate details, and other data on the image, such as acquisition time, 
the remote site, and instantaneous speed, and not the image itself. Images archived at the remote sites 2 can be 
retrieved by the central server 47 when required. 

[01 19] Control of the remote nodes 2 is performed by the remote site user interface 53 which runs on the Sun work- 
55 station 450 connected to the LAN 41 9 of the central server 47. The interface 53 Includes a user tool which communicates 
with a super task of each remote site 2 using a Sun Microsystems Remote Procedure Call (RPC) communications 
protocol. The super task provides a set of procedural functions which can be called by the user tool using the RPC 
protocol, regardless of the location of the workstation 450. The RPC protocol handles data type conversions and align- 
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ment The procedures provided by the super task perform various aotions which together allow complete control of the 
software of a node 2. For example, a parameter file maintains a list of all variables used by the software of the nodes 
2 together with their initial values. The form of the values indicates the variable type, which may be a decimal integer, 
a hexadecimal integer, a floating point value, a character string or boolean value. The variables can be altered by 
adjusting the parameter file, and location of the variables listed in the parameter file is done via a VxWorks systems 
table which contains all global symbols. The user tool, in addition to changing system parameters, can access the 
super task to obtain status and configuration information on each node 2. 

[0120] The super task accepts RPC transactions via both the Transmission Control Protocol (TCP) and the User 
Datagram Protocol (UDP), both of which use the Internet protocol (IP) for transmission of datagrams between computer 
systems. UDP is connectionless protocol which primarily involves multiplexing of datagrams, whereas TCP is a con- 
nection orientated protocol which seeks to ensure data integrity is maintained. The user tool presently uses TCP/IP 
which, together with the RPC protocol, is provided with Sun Microsystem's SunOS operation system and the VxWorks 
real time operating system. To protect against different central stations accessing a remote node and making conflicting 
changes to system parameters, the user tool provides information on the current state of the node software before any 
alteration can be made. 

[0121] The master clocks 354 of the remote sites 2 are synchronised to the clock of the central server 47, and the 
systems 51 and 450 connected to the LAN 491 using a network time protocol (NTP), which is a standard UNIX utility 
normally used to synchronise the clocks of stations on a LAN. The NTP polls the remote sites 2 and on the basis of 
information received from the sites 2 concerning communications between the sites and the server 47, the NTP applies 
offsets to the remote sites 2 so as to synchronise the sites 2 and account for network propagation delays, including 
transient network problems such as link congestion. 

[0122] The vehicle monitoring system is particularly advantageous as it is able to detect and discriminate moving 
vehicles from other objects, and acquire an image of selected vehicles from which they can be identified, using only 
electronic cameras and processing circuitry and software housed at a remote site 2. The system enables automatic 
extraction of licence plate details and does not require road based equipment or markings, the emission of electro- 
magnetic signals or the replacement of film at the node 2. 

[01 23] The system is able to simultaneously track a number of vehicles on multi-lane carriageways and classify them 
by vehicle type. A high resolution image of a vehicle can be obtained over a full traffic lane, the resolution and clarity 
of the invention being sufficient to enable extraction of the licence plate details. The system can operate continuously 
in all conditions where visibility is greater than 100 metres, using infrared imaging techniques. The high resolution 
camera incorporates antiblooming technology to prevent pixel saturation due to vehicle headlights, and the infrared 
flash used is configured so as to be substantially undetectable and inhibit flash dazzle. 

[0124] The system can also be controlled and initialised from a remote central station, with images and data being 
transmitted over a digital communications network. 

[0125] The system can further be used for a number of purposes, such as monitoring tailgating offences, road toll 
collection, and transit lane monitoring. It can also be adapted for red light intersection monitoring. 
[01 26] The system can also be adapted to monitor and acquire images of other moving objects , such as the movement 
of shipping containers within transport depots, and the movement of objects on an assembly line. 



Claims 



1. An object monitoring system comprising camera means (6,8,30,34), characterised in that the camera means is 
adapted to monitor movement of an object (1 8) to predetermine, based on the monitored movement of the object, 
an acquisition time at which an image can be acquired at a predetermined position (22) of said object relative to 
said camera means, and to acquire an image at the predetermined acquisition time and the predetermined position. 

2. An object monitoring system as claimed in claim 1 , wherein said camera means (6,8,30,34) is adapted to detect 
said object (18) and discriminate said object from static objects and other moving objects. 

3 An object monitoring system as claimed in claim 2, wherein the camera means (6,8;30,34) includes video camera 
means (6) for monitoring a respective area (12) in which objects move, and image processing means (30) for 
subtracting a background image of said area from images of said area generated by said video camera means so 
as to produce difference images representative of moving objects in said area. 

4 An object monitoring system as claimed in claim 3, wherein said image processing means (30) includes classifi- 
cation means (80) for forming clusters from parts of said difference images which correspond to the same moving 
object (18). 
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13. Objektuberwachungssystem nach Anspruch 12 miteiner Lichtintensitatsvorrichtung (190) zum Uberwachen eines 
Lichtwerts des Gebiets, wobei der bestimmte Wertebereich, die Analyse der Bereiche, die auf die Bereiche durch 
die Clustererzeugungsvorrichtung (174) angewandte Ausdehnung und die Klassifizierungsdaten in Abhangigkeit 
von dem Lichtwert eingestellt werden. 

14. Objektuberwachungssystem nach Anspruch 13, bei dem die Bildverarbeitungsvorrichtung eine Vorrichtung (180) 
zum Verfolgen der dem Objekt entsprechenden Cluster bei aufeinanderfolgenden Bilder aufweist, die eine Trans- 
formationsvorrichtung zum Transformieren von Koordinaten der Cluster zum Kompensieren eines perspektivi- 
schen Blicks der Kameravorrichtung und eine Vorrichtung zum Vorraussagen der Geschwindigkeit und Position 
des Clusters fur jedes folgende Bild umfaBt. 

15. Objektuberwachungssystem nach Anspruch 14, bei dem die Verfolgungsvorrichtung (180) die Erfassungszeit auf 
der Basis der bestimmten Position (22) und der vorhergesagten Geschwindigkeit und Position des Clusters be- 
stimmt. 

16. Objektuberwachungssystem nach Anspruch 15, bei dem die Kameravorrichtung (6, 8, 30, 34) eine Bilderfassungs- 
kameravorrichtung (8, 34) aufweist, um das Bild des Objektes (18) bei der Erfassungszeit zu erhalten. 

17. ObjektQberwachungssystem nach Anspruch 1 6, bei dem die Bilderfassungskameravorrichtung (8, 34) eingerichtet 
ist, um ein hochauflosendes Bild des Objektes zu erhalten. 

18. Objektuberwachungssystem nach Anspruch 17, bet dem die Videokameravorrichtung (6) ein relativ zu derBilder- 
fassungskameravorrichtung (8, 34) weites Blickfeld (12) hat, die ein eingeschranktes Blickfeld (20) aufweist. 

19. Objektuberwachungssystem nach Anspruch 18 mit einem Infrarotblitz (40), der mit der Bilderfassungskameravor- 
richtung (8, 34) synchronisiert ist, wobei der Energiewert des Blitzes von dem Lichtwert abhangt. 

20. Objektuberwachungssystem nach Anspruch 19, bei dem die Bilderfassungskameravorrichtung (8, 34) eine Bild- 
wahmehmungsvorrichtung (371) und eine Belichtungsregeivorrichtung (40) zum Verhindern einer Sattigung der 
Bildwahrnehmungsvorrichtung als Reaktion auf den Lichtwert aufweist. 1 

21 . Objektuberwachungssystem nach Anspruch 20, bei dem der Blitz eine Vorrichtung zum Unterdrucken der Emission 
sichtbaren Lichts vom Blitz aufweist. 

22. Objektuberwachungssystem nach Anspruch 21 , bei dem die durch die Clustererzeugungsvorrichtung (1 74) ange- 
wandte Ausdehnung zunimmt, wenn der Lichtwert einem Nachtzustand entspricht. 

23. Objektuberwachungssystem nach Anspruch 22, bei dem die Ausdehnung fur Bereiche geringer ist, die Objekten 
entsprechen, die fern von der Kameravorrichtung (6, B) sind. 

24. Objektuberwachungssystem nach Anspruch 23, bei dem die Markierungsvorrichtung (178) das Vergleichen und 
Trennen auf der Basis des Vergleichens von Grenzen Oder Zentren der Cluster von aufeinanderfolgenden Bilder 
durchfuhrt. 

25. Objektuberwachungssystem nach Anspruch 24, bei dem die Differenzbilder gefiltert und eingesetzt werden, um 
das Hintergrundbiid zu aktuaiisieren. 

26. Objektuberwachungssystem nach Anspruch 25 mit einer Vorrichtung (84) zum Ausldsen der Bilderfassungska- 
meravorrichtung (8, 34) bei der Erfassungszeit, die eine Vorrichtung (352) zum Empfangen und Speichem einer 
Zahl einer Abtastlinie, die der Erfassungszeit der Verfolgungsvorrichtung (180) entspricht, eine Vorrichtung (358) 
zum Zahien der Abtastlinien der Bilder und eine Vorrichtung (356) zum Erzeugen eines Auslosesignals fur die 
Bilderfassungskameravorrichtung aufweist, wenn der Zihlwert die Zahi erretcht. 

27. Objektuberwachungssystem nach Anspruch 26, bei dem die Lichtintensitatsvorrichtung (190) ein Histogramm 
(150) von Grauwerten von Bildpunktenfurdievon der Videokameravorrichtung erzeugten Bilder erzeugt und einen 
Tag-, Nacht- Oder Zwielicht-Lichtzustand auf der Basis des Mittelwerts des Histogramms bestimmt. 

28. Objektuberwachungssystem nach Anspruch 27, bei dem der bestimmte Wertebereich auf der Basis des M inimums, 
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des Mittelwerts und des Spitz enwerts des Histogramms bestimmt wird. 

29. Objektuberwachungssystem nach Anspruch 28, bei dem die Messungen die Zirkularitat und die Uberdeckung der. 
Bereiche umfassen. 

5 

30. Objektuberwachungssystem nach einem der Anspruche 1 bis 29 mit einer Erkennungsvorrichtung (51) zum Ver- 
arbeiten des erfaflten Biides, urn Informationen zu erhalten, die das Objekt (18) identrfizieren. 

31 . Objektuberwachungssystem nach Anspruch 30 mit mehreren Kameravorrichtungen (6, 8, 30, 34) zum Uberwachen 
10 jewei tiger Gebiete, die eingerichtet sind, urn miteinander zu kommunizieren, urn informationen uber das Objekt 

(18) zu ubertragen. 

32. Objektuberwachungssystem nach Anspruch 30 oder 31 mit mehreren Kameravorrichtungen (6, 8, 30, 34) zum 
Uberwachen jeweiliger Gebiete, die eingerichtet sind, urn mit einer Zentralstation (47) zu kommunizieren, urn In- 

15 formationen uber das Objekt (18) zu ubertragen. 

33. Objektuberwachungssystem nach Anspruch 32, bei dem die Informationen uber das Objekt (18) von zumindest 
zwei Kameravorrichtungen (6, 8, 30, 34) erfaBt werden, und die Informationen eingesetzt werden konnen, urn die 
Zeit zu bestimmen, die das Objekt braucht, urn sich zwischen zumindest zwei der Gebiete zu bewegen. 

20 

34. Objektuberwachungssystem nach Anspruch 33, bei dem die Zentralstation (47) eine Femsteuervorrichtung (53) 
zum Steuem der Kameravorrichtung von der Zentralstation aus aufweist. 

35. Objektuberwachungssystem nach Anspruch 34, bei dem die Zentralstation (47) und die mehreren Kameravonich- 
25 tungen (6, 8, 30, 34) jeweilige Telekommukikationskontroller (55) aufweisen und unter Verwendung eines digitalen 

Telekommunkationsnetzwerkes (45) kommunizieren. 

36. Objektuberwachungssystem nach Anspruch 35 mit einer Vorrichtung (49) zum Archivieren der Informationen und 
zum Gestatten eines anschlieftenden Zugriffs darauf. 

30 j 

37. Objektuberwachungssystem nach Anspruch 36, bei dem die Informationen erfaftte Bilder des Objektes und die 
Erfassungszeiten umfassen, und die Zentralstation (47) die Erkennungsvorrichtung (51) aufweist. 

38. Objektuberwachungssystem nach Anspruch 36, bei dem die Informationen identifizierende Informationen und die 
35 Erfassungszeiten erfaSter Bilder des Objektes (18) umfassen, und mehrere Erkennungsvorrichtungen (51) an die 

mehreren Kameravorrichtungen (6, 8, 30, 34) jeweils bei den Orten der mehreren Kameravorrichtungen (6, 8, 30, 
34) angeschlossen sind. 

39. Objektuberwachungssystem nach Anspruch 37 oder 38, bei dem die Erkennungsvorrichtung (51) eingerichtet ist, 
40 um das erfaBte Bild zu verarbeiten, urn Bildpunktezu ermttteln, die furcharakteristische Bildpunkte kennzeichnend 

sind, die das Objekt (18) identifizieren. 

40. Objektuberwachungssystem nach einem der Anspruche 1 bis 39, bei dem das Objekt (18) ein Fahrzeug ist. 

45 41. Objektuberwachungssystem nach Anspruch 40 in Abhangigkeit von Anspruch 30, bei dem die Erkennungsvor- 
richtung eine Vorrichtung zum Ermitteln eines Nummernschilds in dem Bild und eine Vorrichtung zum Bestimmen 
der Schriftzeichen des Nummernschilds umfaGt, wobei die Schriftzeichen die identifiziemden Informationen um- 
fassen. 

so 42. Objektuberwachungssystem nach Anspruch 41 in Abhangigkeit von Anspruch 6, bei dem der bestimmte Wertebe- 
reich Bildpunktwerte abdeckt, die durch Schatten des Fahrzeugs erzeugt werden. 

43. Objektuberwachungssystem nach Anspruch 42 in Abhangigkeit von Anspruch 8, bei dem die ungultigen Bereiche 
Fahrlichtreflektionen entsprechen, die von Fahrzeugen oder Fahrbahnmarkierungen in dem Gebiet erzeugt wer- 

55 den. 

44. Objektuberwachungssystem nach einem der Anspruche 40 bis 43, bei dem das Fahrzeug ein groBes Fahrzeug, 
zum Beispiel ein Bus oder Laster, ist. 
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45. Objektiiberwachungssystem nach einem der Ansprtiche 1 bis 44 bei dem das ObiPkt rim ain c u 

bei dem die Kameravorrichtung (6, 8, 30 34) einqerichtet tat umri-J p!k ^ (18) " fahrzeu 3 ,st - und 

eine Segmentierungsvorrichtung (77, 78) zum Verarbeiten. der Differenzbilder urn Bilder dp, Rflroi ,K 

SS die,0rBereichekennzeiohnendsind ' die ^ 

eine Klassifizierungsvorrichtung (80) zum Verarbeiten der Bilder des Bereichs wobei die km*., 
nchtung eine Vorrichtung (176) zum Analysieren der Form der Bef*^^^^T B ' W !i 
ungQrtigar Bereiche auf der Basis der Analyse, eine C\u S t e Sr^man 7 T^^ ^ Und 



Revendicatlons 

1. 



2. 



3. 



Systemede surveillance d'un objet comprenant des moyens formes de cameras «5 8 30 ™ 
que les moyens formes de cameras sont adaotes oour rnntrAior mLmaT ( ' ' 34) * caract6ris6 en ce 
predeterminer, sur la base du ! d6p !* Cements d ' un ob i et d B) de maniere a 



EP 0 631 683 B1 



4. Systeme de surveillance d'un objet selon la revendication 3, da*ns lequel lesdits moyens (30) de trailement d'images 
incluent des moyens de classification (80) pour former des regroupements a parti r de parties desdites images de 
difference, qui correspondent au meme objet mobile (18). 

5. Systeme de surveillance d'un objet selon la revendication 4, dans lequel lesdits moyens (30) de traitement d'images 
traitent chaque regroupement pour determiner s'il correspond audit objet (18) et determine, iedit instant d'acqui- 
sition. 

6. Systeme de surveillance d'un objet selon la revendication 5, dans lequel Iedit moyen (30) de traitement d'images 
filtre lesdites images de difference pour ne pas prendre en compte des pixels dans une gamme predetermin6e de 
niveaux (79). 

7. Systeme de surveillance d'un objet selon ia revendication 6 dans lequel lesdits moyens (30) de traitement d'images 
incluent des moyens de segmentation (76) pour traiter lesdites images de difference de man i ere a produire des 
images segmentees qui incluent des regions correspondant a des parties d'objets mobiles dans ladite zone, qui 
produisent au moins un niveau de lumiere predetermine dans lesdits moyens formes de cameras. 

8. Systeme de surveillance d'un objet selon la revendication 7, dans lequel lesdits moyens de classification (80) 
analysent et produisent des mesures de ia forme desdites regions et, sur la base de ('analyse et des mesures, 
determined des regions valables et des regions non valables ne devant pas etre prises en compte. 

9. Systeme de surveillance d'un objet selon la revendication 8, dans lequel lesdits moyens de classification (80) 
comprennent des moyens de regroupement (174) pour produire lesdits regroupements, chaque regroupement 
comprenant des regions valables, qui sont considerees comme correspondant a un objet, lesdites regions etant 
agrandies pour determiner si des regions se chevauchent entre elles et doivent etre combinees entre elles pour 
former un regroupement. . 

10. Systeme de surveillance d'un objet selon la revendication 9, dans lequel lesdits moyens de classification (80) 
comprennent des moyens d'etiquetage (1 78) pour affecter une etiquette a chaque regroupement pour chaque 
image de maniere a identifier des regroupements respectifs, et pour mettre en correspondence et separer des 
regroupements dans des images successives pour determiner si des etiquettes doivent etre heritees ou si de 
nouvelles etiquettes doivent dtre affectees. 

11. Systeme de surveillance d'un objet selon la revendication 10, dans lequel lesdits moyens de classification (80) 
sont adaptes pour classer lesdits regroupements comme correspondant a des objets predetermines par compa- 
raison de caracteristiques desdits regroupements avec des donnees de classification dudit systeme, de telle sorte 
que lesdits moyens de classification sont adaptes pour identifier un regroupement correspondant audit objet (1 8). 

12. Systeme de surveillance d'un objet selon la revendication 11, comprenant des moyens (190) pour maintenir un 
histogramme (150) desdites caracteristiques pour des objets contr6les par lesdits moyens formes de camera (6, 
8, 30, 34) et pour ajuster lesdites donnees de classification sur la base dudit histogramme. 

13. Systeme de surveillance d'un objet selon la revendication .12, comprenant des moyens (190) de reglage de I'in- 
tensite lumineuse servant a controler un niveau d'eciairement de ladite zone, et dans iequel ladite gamme pr6d6- 
terminee de niveaux, ladite analyse desdites regions, ('extension appliquee auxdites regions par lesdits moyens 
de regroupement (174) et lesdites donnees de classification sont r6gl6es en fonction dudit niveau d'eciairement. 

14. Systeme de surveillance d'un objet selon la revendication 13, dans lequel lesdits moyens de traitement d'images 
comprennent des moyens (1 80) pour realiser le suivi du regroupement correspondant audit objet dans des images 
successives, comprenant des moyens de transformation pour transformer des coordonnees dudit regroupement 
pour compenser une vue en perspective desdits moyens formes de cameras, et des moyens pour p red ire la Vitesse 
et ia position dudit regroupement pour chacune des images successives. 

1 5. Systeme de surveillance d'un objet selon la revendication 1 4, dans lequel lesdits moyens de suivi (1 80) determinent 
1'instant d'acquisition sur la base de ladite position pr6d6termin6e (22) et de la vitesse et de ia position predites 
dudit regroupement. 

16. Systeme de surveillance d'un objet selon la revendication 15, dans lequel lesdits moyens formes de cameras (6, 
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is z ss szssr 34) - °— "-<- -i - «=• 

captage d'images, qui possedent un champ d'observation limite (20). cameras (8, 34) de 

21. Syst&me de surveillance d'un objet selon la revendication 20 dans leauel indito i.™ u, • 

pour empecher remission d'une lumiere visible par la lampe flash P ' nClUt des ™y ens 

23. Systeme de surveillance d'un objet selon la revendication 22, dans leauel ladrte extent™ « ■ 

regions correspondent a des objets distants desdits ^sl^X J^™? * " 1 * Sr ' eUre P °" r deS 

M ' cIn 6 r^! S m T ei ' lanCe d ' Un T Se ' 0n 18 revendication 23- "ans lequel lesdits 'moyens d'etiquetege (1 78) exe- 

~rS: P rnts 0 ^ 

25 - K^SCSS; XatSSSEX." 1 " Iequel lesdi,es ima9es de < m ™ ™ 

d'images lorsque ledit comptage atteint ledit nombre. 6raS de captage 

27. Systeme de surveillance d'un objet selon la revenriiratinn or w q „* • ^ 

* rs: ±a sassassr * - — — — — - 

30. Systeme de surveillance d'un objet selon I'une queiconque des revendications 1 a 2Q rnmnm„ Q n, m 

31. Systeme de surveillance d'un objet se.cn .a revendication 30. comprenen. une pluralrte desdits moyens formes 
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de cameras (6, 8, 30, 34) pour surveiller des zones respectives.'et adaptes pour communiquer entre eux de maniere 
a transferer une information concernant ledit objet (18). 

32. Systeme de surveillance d'un objet selon la revendication 30 ou 31 , comprenant une pluralite de moyens formes 
5 de cameras (6, 8, 30, 34) pour surveiller des zones respectives, et adaptes pour communiquer avec un poste 

central (47) de maniere a transferer une information concernant ledit objet (18). 

33. Systeme de surveillance d'un objet seion la revendication 32, dans lequei ladite information concernant ledit objet 
(18) est acquise a I'aide d'au moins deux desdits moyens formes de cameras (6, 8, 30, 34) et ladite information 

w peut etre utilisee pour determiner la duree, que met ledit objet a se deplacer entre au moins deux desdites zones. 

34. Systeme de surveillance d'un objet seion la revendication 33, dans lequei ledit poste centra) (47) inciut des moyens 
de telecommande (53) pour commander lesdits moyens formes de cameras a partir dudit poste central. 

is 35. Systeme de surveillance d'un objet selon la revendication 34, dans lequei ledit poste central (47) et ladite pluralite 
de moyens formes de cameras (6, 8, 30, 34) comprennent des dispositifs respectifs (55) de commande de tele- 
communication et communiquent en utilisant un reseau de telecommunication numerique (45). 

36. Systeme de surveillance d'un objet selon la revendication 35, comprenant des moyens (49) pour archiver ladite 
20 information, et permettre un acces ulterieur a cette demiere. 

37. Systeme de surveillance d'un objet selon la revendication 36, dans lequei ladite information inciut des images 
acquises dudit objet et ies instants d'acquisition, et ledit poste central (47) inciut lesdits moyens de reconnaissance 
(51). 

25 

38. Systeme de surveillance d'un objet selon la revendication 36, dans lequei ladite information inciut ladite information 
d'identification et Ies instants d'acquisition d'images acquises dudit objet (18), une pluralite desdits moyens de 
reconnaissance (51) sont connectes respectivement a ladite pluralite de moyens formes de cameras (6, 8, 30, 
34), au niveau des sites de ladite pluralite de moyens formes de cameras. 

30 { 

39. Systeme de surveillance d'un objet seion la revendication 37 ou 38, dans lequei lesdits moyens de reconnaissance 
(51 ) sont adaptes pour traiter ladite image acquise pour reperer des pixels representatifs de pixels caracteristiques 
identifiant ledit objet (18). 

35 40. Systeme de surveillance d'un objet selon rune quelconque des revendications 1 a 39, dans lequei ledit objet (1 8) 
est un vehicule. 

41 . Systeme de surveillance d'un objet selon la revendication 40, consideree comme dependante de la revendication 
30, dans lequei lesdits moyens de reconnaissance comprennent des moyens pour reperer une plaque d'immatri- 

40 culation dans ladite image et des moyens pour determiner Ies caracteres de la plaque d'immatriculation, lesdits 

caracteres comprenant ladite information d'identification. 

42. Systeme de surveillance d'un objet selon la revendication 41 , consideree comme dependante de la revendication 
6, dans lequei ladite gamme predeterminee de niveaux englobe Ies niveaux de pixel produits par des ombres dudit 

45 vehicule. 

43. Systeme de surveillance d'un objet selon la revendication 42, consideree comme dependante de la revendication 
8, dans lequei lesdites regions non valables correspondent a des reflexions de phares, produites par lesdits ve- 
hicules ou des marquages routiers sur la chaussee a I'interieur de ladite zone. 

50 

44. Systeme de surveillance d'un objet selon i'une quelconque des revendications 40 a 43, dans lequei ledit vehicule 
est un vehicule de grande taille, comme par exemple un autobus ou un camion. 

45. Systeme de surveillance d'un objet selon I'une quelconque des revendications 1 a 44, dans lequei ledit objet (18) 
55 est un vehicule et dans lequei lesdits moyens formes de cameras (6, 8, 30, 34) sont adaptes pour surveiller ledit 

vehicule pour detecter une infraction a la loi et captent une image dudit v6hicule audit instant predetermine en 
reponse a la detection de ladite infraction. 
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46. Systeme de surveillance d'un objet selon la revendication 45, comprenant des moyens da reconnaissance (51) 
pour trarter lesdites .mages pour obtenir una information identifiant ledit vehicule. ™=°nna 1S sance (51 ) 

47 ' Sel ° n ' a reven "" 1 • ^ns lequel les moyens formes de cameras (6, 8 30 34) pro- 

2^S2Sr " ima9e ^ ° bje ' **"« * -vens^da 

des moyens (70, 72, 74, 76) pour soustraire une image darriere-plan desdites images de ladite zone oour 
produire des .mages de difference representatives d'objets mobiles dans ladite zona P 

IZT 0 ^"? d6 ! e gmen,ation < 77 ' 78 > P° ur ^iter lesdites images de difference pour produire des images 
representees de regions correspondant a des parties desdits objets mobiles dans ladrte zone 
des moyens de classffication (80) pour trailer lesdites images de regions, lesdits moyens de classification 
mcluant ,« moy ens (1 76) pour analyser la forme desdites regions et. sur la base de I'anahyse deSSes 
regions valables et des regions non valables, des moyens de regroupement (176) pour produire sTr la bSe 
desd.tes reg.ons valables, les regroupements correspondant a des objets respect ptZteZteZ^™ 
b.l« 'fdesmoyena^^pourciasserleadasregroupemen^ 

desd.ts regroupements a des donnees de classification dudit systems pour determiner si ZSZT 
pements correspond audit objet predetermine; et 9 
des moyens de suiyi (180) pour realiser le suivi de run desdits regroupements correspondant audit obiet 

ESS.'™ ^ d ' aCqUiSi,i ° n d ' imageS P ° Ur taC ^ i0n de ,adita £1 2 oojet 

48. Systeme de surveillance dun objet selon la revendication 47, dans lequel lesdits moyens (30) de traitement d'ima- 

,ma9es de di,,erence pour ne pas prendre en comp,e des pixei " *» « 

49 ' TT?? B SUrveillance d ' un ob j et selon la revendication 48, dans lequel lesdites parties tf objets mobiles corres- 
pondent a au mens un n.veau de lumiere predetermine recu au milieu desdits moyens forme's da 

5 °' ftend^ .«H 1 l" V8 f, nCe Se '° n ' a revendication 49 > da ™ lequel lesdits moyens de classification (80) 

Supemel ^ ** " ' eSditeS reS,i ° nS d ° K ' ent 8tre combine <* P™" ™X2 
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